---
title: "ONLINE APPENDIX"
subtitle: "Judicial Decisions on Electronic Control Devices and Police Escalation of Force"
author: "Courtenay R. Monroe, Sophia Hatz and Kristine Eck"
date: "`r format(Sys.time(), '%d %B %Y')`"
header-includes:
  - \usepackage{float}
  - \floatplacement{figure}{!H}
output:
  pdf_document:
    toc: yes
    toc_depth: '2'
    number_sections: true
    extra_dependencies: ["float"]
  fontsize: 12pt
  geometry: margin=1in
always_allow_html: true
---

```{r setup, include=FALSE}
# set working directory

# set seed
set.seed(123765896)
# install and call packages 

library(haven)
library(readxl)
library(tidyverse)
library(plyr)
library(dplyr)
library(ggplot2)
library(fixest)
library(stargazer)

# clear environment
rm(list = ls())

# global knitr options
knitr::opts_chunk$set(eval=TRUE,cache=F, include=F, echo=F, error=F,message=FALSE,fig.align = "center", warning=FALSE,width=180,results='markup',fig.path="",fig.height = 5, fig.width = 8,fig.pos = "!H", out.extra = "",dev = c('png'),fig=FALSE,pointsize=10, comment=c(""), collapse=FALSE)

load("JD_replication_data.RData")



```

\newpage{}
# Data sources and coding of variables
We combine data on U.S. Circuit Court of Appeals decisions related to Electronic Control Devices (ECDs) and data on civilians shot and killed by police. We also include data on crime rates and economic and demographic indicators. Our panel data set covers 50 US states (plus the District of Columbia) during 2000-2019. The unit of analysis is the state-year.

## Independent variable

Data on ECD jurisprudence is publicly available and provided by Americans for Effective Law Enforcement (AELE).^[Americans for Effective Law Enforcement (AELE), "Electronic Control Weapons AELE Case Summaries", August 6, 2019, Distributed by AELE, https://aele.org/law/Digests/ECWcases.html] For each court case in the AELE data, we know the circuit court that issued the decision, the corresponding states that were affected, the year and month in which the decision was issued, and whether the decision was restrictive or non-restrictive of ECDs. During the 20 years in our study, there were 163 judicial decisions related to ECDs. Of these, 70 (42%) were restrictive of ECDs. 

From these data, we code two treatment variables. The first is *restrictive decision*, a binary variable coded 1 if a state's circuit court issued (at least one) restrictive judicial decision on ECDs in a given state-year. It is coded 0 if no judicial decision was issued, or if only non-restrictive decisions were issued. The second treatment variable is *restrictive decision count*, indicating the number of restrictive judicial decisions on ECDs in a given state-year. 


## Dependent variable

The dependent variable *police shooting deaths* is the total number of deaths incurred by police via gunshot, excluding gunshot deaths due to suicide. We use data complied by Fatal Encounters, covering 50 US states (plus the District of Colombia) during 2000-2019.^[Fatal Encounters, "FE Database", September 18, 2020. Distributed by Fatal Encounters, https://fatalencounters.org/]


## Control variables

We include data on time-varying control variables. 

First, we include indicators on crime from the Federal Bureau of Investigation's (FBI) Uniform Crime Reporting (UCR) Program.  We use data collated by Jacob Kaplan, available through the Inter-university Consortium for Political and Social Research (ICPSR).^[Kaplan, Jacob. "Jacob Kaplan's Concatenated Files: Uniform Crime Reporting Program Data: Offenses Known and Clearances by Arrest (Return A), 1960-2020". September 27, 2021, Distributed by Interuniversity Consortium for Political and Social Research, https://doi.org/10.3886/E100707V17-100064] The UCR data include data on crimes reported to the police. We choose two primary indicators of crime: homicide (murder and non-negligent manslaughter) and property crime. Police agencies submit to the UCR on a voluntary basis, which makes it difficult to distinguish between a true zero (no crime) and no report. In order to attribute true zeros, we only include agencies which consistently reported (each month of the year) to the UCR during 2000-2019. We aggregate the data to the state-year level and calculate rates per 100,000 residents, using population data in the UCR. We imputed missing data in the UCR variables using K-nearest neighbor matching.

- Homicide rate per 100,000 residents (2000-2019)
- Property crime rate per 100,000 residents (2000-2019)

Second, we include state-level economic indicators and demographic characteristics from the University of Kentucky Center for Poverty Research (UKCPR):^[University of Kentucky Center for Poverty Research, "UKCPR National Welfare Data, 1980-2020", February 2022, distributed by UKCPR, http://ukcpr.org/resources/national-welfare-data.]

- Unemployment rate (2000-2019)
- Poverty rate (2000-2019)
- Total state population (2000-2019)

## Summary statistics
Table 1 shows summary statistics for the variables in our analysis. In estimation, we use the natural log of state population because of its skewed distribution. Figure 1 shows the distribution of the variables used in analysis. 


```{r}
names(jd.df)


treatvars <- c("restrictive_aele","restrictive_count_aele")
depvars <- c("police_shoot_fe")
covars <- c("homicide_rate_ucr","property_crime_rate_ucr","total_arrests_rate_ucr","unemp_rate_ukcpr","poverty_rate_ukcpr","population_ukcpr_log","pctnonwhite_cspp")

jd.df %>% 
  dplyr::group_by(year) %>% 
  dplyr::summarise(across(all_of(covars),function(x) sum(!is.na(x))))

# excluding variables with poor coverage
covars <- c("homicide_rate_ucr","property_crime_rate_ucr",  "unemp_rate_ukcpr","poverty_rate_ukcpr","population_ukcpr_log")

vars <- c(treatvars, depvars, covars)

variable.labels <- c("Restrictive decision","Restrictive decision count", 
"Police shooting deaths", 
"State homicide rate","State property crime rate",
"State unemployment rate","State poverty rate","State population (log)")
```


```{r, include=TRUE, results='asis'}
vtable::sumtable(jd.df[,vars],labels=variable.labels,summ=c('notNA(x)', 'mean(x)', 'sd(x)', 'min(x)', 'max(x)','propNA(x)'),factor.percent=F,digits=2,summ.names=c("N","Mean","Sd","Min","Max","Prop. missing"),out="latex") 
```




```{r,summary_distributions, fig.height=6, include=TRUE}

jd.df %>% ungroup %>% select(all_of(vars)) %>% tidyr::gather(variable, value) %>% mutate(variable=factor(variable,levels = vars,labels=variable.labels)) %>%
  ggplot(aes(x = value)) +
  geom_histogram() +
  facet_wrap(~ variable,scales="free",ncol=2) +
  ggtitle("Figure 1: Variable distributions") +
  xlab("") +
  theme_bw()



```





# Research design

We employ a generalized difference-in-difference (DID) design. DID designs are used to estimate the effect of an intervention that affects some units but not others. In DID designs, the effect of the treatment is the difference in before and after levels of the outcome across treated- and not-treated units. 

Our application of DID is complicated by two characteristics of the treatment. The first is "differential treatment timing" or "staggered treatment adoption": units receive treatment at different points in time (De Chaisemartin and d'Haultfoeuille 2020, Goodman-Bacon 2021). To illustrate the problem of differential treatment timing, Figure 2 shows the timing of restrictive judicial decisions from 2000 to 2019. In 2005, there was a restrictive decision by the First Circuit Court, affecting Alabama, Florida and Georgia; and a restrictive  decision by the Fifth Circuit Court, affecting Louisiana, Mississippi, and Texas. These six states were "treated" in 2005. A different set of states was treated in 2007 and so on. 

With differential timing, researchers typically employ Two-way Fixed Effects (TWFE) or First Difference (FD) as a DID estimator. Recent methodological work, however, has shown that TWFE and FD can return estimates that are severely biased, especially when treatment effects are heterogenous across groups and over time, as tends to be the case with differential timing (e.g., Borusyak, Jaravel and Spiess 2021, Callaway and Sant'Anna 2021, De Chaisemartin and d'Haultfoeuille 2020, Goodman-Bacon 2021, Sun and Abraham 2021). However, several alternative estimators are robust to treatment effect heterogeneity (e.g., Borusyak, Jaravel and Spiess 2021, Callaway and Sant'Anna 2021, De Chaisemartin and d'Haultfoeuille 2020, Liu and Xu 2022).

A second challenge we observe in Figure 2 is treatment "reversibility": treatment status switches on and off (Callaway and Sant'Anna 2021, De Chaisemartin and d'Haultfoeuille 2020). Alabama is treated by restrictive decisions in 2005, untreated in 2006, treated again in 2007 and so on. Although the reversibility of restrictive decisions complicates the application of a DID research design, this is of substantive interest to us. We hypothesize that these back-and-forth switches in judicial decisions lead to increases in instances of deadly force. Thus, operationalising switches in judicial decisions is key to testing our theoretical argument. To this end, we choose the $DID_M$ estimator from De Chaisemartin and d'Haultfoeuille (2020), one of the few DID estimators which is appropriate in the presence of differential treatment timing and when treatment is reversible. 



```{r}
table(jd.df$restrictive_aele,jd.df$year,useNA="ifany")
# check for never-treated
unique(jd.df$state[sum(jd.df$restrictive_aele)==0])


st <- jd.df %>% filter(restrictive_aele==1) %>% 
  select(circuit,restrictive_aele,year,state,state_name) %>% 
  arrange(year,circuit,rev(state)) %>% 
  group_by(year) %>% dplyr::mutate(ntr=cumsum(restrictive_aele)) 
#use rev(value) to get alphatical order (from top down) in plot
st[1:20,]
tail(st)
sapply(st,class)


```
\newpage{}
**Figure 2**: Timing of restrictive judicial decisions across US states during 2000-2019
```{r, treatment_timing, include=TRUE}
ggplot(st, aes(x= year, y= ntr, label=state, color=circuit)) +
  geom_point(aes(color=circuit),size=-3,show.legend = TRUE) +
  geom_text(aes(y= ntr,label=state),hjust=0.5, vjust=0.2, size=3,show.legend = FALSE) + 
  ylab("Treated states\nTreatment = restrictive decision") +
  scale_colour_discrete(name  ="Circuit court",
                            breaks=c(1:11),
                            labels=c(1:11)) +
  guides(color = guide_legend(override.aes = list(size = 2))) +
  theme_bw()

```



```{r}

#Switching treatment status is fairly common: on average states switch 4 times during 2000-2019. 13.7% of states switch twice; 56.9 % of states switch 4 times,5.9% switch 6 times and 15.7% of states switch 7 times. Only 4 states do not switch treatment status at any point: CT, NY, VT in circuit 2 and DC, in circuit 12. 

st.sq.t <- jd.df %>% group_by(state) %>% dplyr::summarise(sum=sum(abs(restrictive_aele_diff),na.rm=TRUE)) %>% dplyr::mutate(sum.f=factor(as.character(sum))) 
mean(st.sq.t$sum,na.rm=TRUE)
round(prop.table(table(st.sq.t$sum.f)),3)

unique(st.sq.t$state[st.sq.t$sum==0])


```


```{r}

#States could have as many as five restrictive judicial decisions in a single year, and this is not super uncommon (7.8% of states experienced 5 restrictive decisions in a single year at least once). 

summary(jd.df$restrictive_count_aele)

st.rn.t <- jd.df %>% group_by(state) %>% dplyr::summarise(max=max(restrictive_count_aele,na.rm=TRUE)) %>% dplyr::mutate(max.f=factor(as.character(max))) 

round(prop.table(table(st.rn.t$max.f)),3)

```



```{r}

table(jd.df$restrictive_aele_diff,useNA="ifany")
jd.df[1:20,c("state","year","judicial_decision_aele","restrictive_aele_diff")]
jd.df[jd.df$circuit==8 & jd.df$state=="AR",c("state","year","judicial_decision_aele","restrictive_aele_diff")]


# 1 is a joiner: treatment is 1 in t, was 0 in t-1
# - 1 is leaver: treatment is 0 in t, was 1 in t-1
# 0 is a stable unit: no change in treatment across consecutive years t and t-1

# check for stable units across all consecutive years
table(jd.df$restrictive_aele_diff,jd.df$year,useNA="ifany")



```

```{r}
st.diff <- jd.df %>% filter(restrictive_aele_diff!=0) %>% 
  select(circuit,restrictive_aele_diff,year,state,state_name) %>% 
  arrange(year,circuit,rev(state)) %>% 
  dplyr::mutate(tr= 1) %>% 
  group_by(year) %>% dplyr::mutate(ntr=cumsum(tr)) %>%
  dplyr::mutate(switch=factor(restrictive_aele_diff,levels=c(-1,1),labels=(c("switching out","switching in"))))
#use rev(value) to get alphatical order (from top down) in plot
st.diff[1:20,]
tail(st.diff)
sapply(st.diff,class)


```

The $DID_{M}$ estimator captures the effect of switches in treatment status by comparing the evolution of outcomes among units that switch treatment status across a pair of consecutive time periods to units that do not switch in the same period. In our setting, this estimator thus compares the evolution of **police shooting deaths**, among states whose status on the treatment variable 
*restrictive decision* changes from one year to the next to the evolution of outcomes among states whose treatment status remains stable. The parameter of interest is the average of these comparisons across all pairs of consecutive time periods and all values of the treatment. When design assumptions hold, $DID_{M}$ is an unbiased and consistent estimator of the average treatment effect among the switchers at the time period when they switch.\footnote{To be clear, the independent (treatment) variable is *restrictive decision*; coded 1 if a restrictive decision was issued in the state-year, and 0 if no decision or only non-restrictive decisions were issued. To construct comparisons of "switchers" and "stable units", the $DID_{M}$ estimator uses the first difference of the treatment variable *restrictive decision*. This first difference captures switches emanating from the courts in that it indicates a reversal, from year-to-year, in whether a court has restricted ECD-use.}

Figure 3 shows the timing of switching treatment status across U.S. states. The figure also shows the two different kinds of switches: states can "switch in" to restrictive decision (from no decision or only non-restrictive decisions in the previous year), and states can "switch out" of a restrictive decision. The $DID_{M}$ estimator calculates a weighted average of both types of switches in treatment status, capturing switches in both directions. This is important, as we theorize about the effect of \emph{back-and-forth} changes in judicial signals. The $DID_M$ estimator is thus theoretically appropriate for testing our hypothesis about switches from judicial decisions. It is also methodologically appropriate given that challenge of "reversibility".

\newpage{}
**Figure 3**: Timing of switches in restrictive judicial decision across US states during 2000-2019
```{r, fd_treatment_timing, include=TRUE}
ggplot(st.diff, aes(x= year, y= ntr, label=state, color=switch)) +
  geom_point(aes(color=switch),size=-3,show.legend = TRUE) +
  geom_text(aes(y= ntr,label=state),hjust=0.5, vjust=0.2, size=3,show.legend = FALSE) + 
  ylab("Switching states\nTreatment = restrictive decision") +
  scale_colour_discrete(name  ="Type of switch") +
  guides(color = guide_legend(override.aes = list(size = 2))) +
  theme_bw()


```

Note: our data meet the Stable Groups Assumption required for the $DID_{M}$ estimator. $DID_{M}$ does not require never-treated units or not-yet treated units for comparison, but it does require units in each pair of consecutive years whose treatment status does not change to serve as a control group. Because there are no years with 51 treated units in our data, our data meet this assumption.



# Main results

**Table 2** shows $DID_M$ estimates for our binary and count treatments, with and without covariates. 
```{r, include=TRUE, results='asis'}

# run _Run_all_Stata_second.do in Stata


# import results from Stata

t2 <- readxl::read_xlsx("table2.xlsx") %>% mutate(estimate= ifelse(pval<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimate,se,pval,N) %>% data.frame(row.names=c("m1: Restrictive decision", "m1b: with covariates","m2: Restrictive decision count","m2b: with covariates"))

t2 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r") %>%
  kableExtra::kable_styling(latex_options = "hold_position")
```
Notes: This table reports $DID_{M}$ estimates: the average treatment effect among states that switch treatment status, at the year when the switch occurs. Model 1 uses a binary treatment variable, indicating at least one restrictive judicial decision on ECD-use in a state-year. In model 2 the treatment variable is the count of restrictive decisions in a state-year. Models 1b and 2b include the following covariates: homicide rate and property crime rate. The dependent variable in all models is the count of police shooting deaths in a state-year. Standard errors are calculated via bootstrapping (200 repetitions) and are clustered by state. Observations denotes number of observations used in the estimation of effect: the number of first differences of the outcome and of the treatment. * indicates significance at the 95% confidence level.

```{r}
# to copy-paste in article text
t2 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r") 

```



# Pre-treatment trends in police violence

Following the basic logic of the DID design, interpreting the effects we observe as causal relies on the extent to which we satisfy the parallel trends assumption---the assumption that, in the absence of treatment, treated and control units would evolve in similar ways. In our context, we need to assume that trends in police shooting deaths across states switching status on restrictive decisions and states remaining stable would have been similar in the absence of changes in judicial decisions. Fundamentally, this assumption implies that treatment is as-if-randomly assigned.

One of the benefits of testing our expectations at the level of the U.S. state rather than the Circuit Court of Appeals is that it allows us to better exploit as-if random assignment of our treatment variable. First, using the state-year allows us to investigate the effect of judicial decisions in states from which the original case did not originate. This is appealing because states in which ECD cases originate may be systematically different than states where ECD cases do not. 
Second, numerous Circuit Court decisions in which ECD decisions are made are multiple weapons claims: decisions in which \emph{Graham} is applied not only to ECDs, but also to other weapons. We are able to exploit the fact that most claims are not specifically about ECDs as additional source of random variation.

Although the parallel trends assumption and our claims about as-if-random assignment are untestable, we can look at pre-treatment trends in police shooting deaths to assess whether treated and control states are on similar trajectories absent treatment. We calculate the placebo estimators proposed in De Chaisemartin and d'Haultfoeuille (2020), which compare the outcome trends of switchers and stable units in the years before the switchers switch. We calculate the coefficients of placebo estimators measured at 1, 2, 3, 4, and 5 years prior to a switch in treatment status. 


```{r}
# import results from Stata

m1p <- readxl::read_xlsx("m1placebo.xlsx") %>% select(estimate,se) %>% dplyr::mutate(t=(c(0,-1,-2,-3,-4,-5)),IDV= c("Restrictive decision"), DV= c("Police shooting deaths"))
m2p <- readxl::read_xlsx("m2placebo.xlsx") %>% select(estimate,se) %>% dplyr::mutate(t=(c(0,-1,-2,-3,-4,-5)),IDV= c("Restrictive decision count"), DV= c("Police shooting deaths"))

placebos <- rbind(m1p,m2p) %>% dplyr::mutate(DV=factor(DV,levels=c("Police shooting deaths")),IDV=factor(IDV,levels=c("Restrictive decision","Restrictive decision count")))

```

Figure 4 shows these pre-treatment trends. In most cases, the placebo estimates are close to zero, and the 95\% confidence intervals generally include zero. But we do note a significant estimate at 3 years prior. At the same time, we struggle to think of a systematic difference that would manifest only three years prior to a switch. Assuming there were systematic differences across switchers and stable units, we might expect the coefficients of the placebo estimators to be steadily increasing or steadily decreasing in the years prior, indicating a pre-existing divergence in police shooting deaths. Our conclusion from the pre-treatment trends is that we should be cautious when interpreting the estimated effect as causal.

\newpage
**Figure 4**: Placebo $DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths

```{r pre_trends, include=TRUE}
ggplot2::ggplot(placebos, aes(x=t, y=estimate)) + 
    geom_point() + 
    geom_errorbar(aes(ymin=estimate-se*1.96, ymax=estimate+se*1.96), width=.1) +
    geom_line() +
    geom_hline(yintercept=0, linetype="dashed",color="grey") +
    scale_y_continuous(position="right") +
    facet_grid(IDV ~ DV,switch="y") + xlab("relative time to treatment (years before switch)") + ylab("DID_M estimate") + theme_bw()

```

# Additional results
## Alternative DID estimators

This section shows results using the more commonly-used first difference (FD) and Two-way Fixed Effect (TWFE) DID estimators. FD models are linear regressions including first differences of all variables and fixed effects for time (year, in our case). These regressions are conceptually and analytically closest to the $DID_M$ models, which also rely on first differences. TWFE models are linear regressions which include fixed effects for time (year) and cross-sectional unit (state). These are popular DID estimators, although recent methodological work has shown that TWFE and FD can return estimates that are severely biased, especially when treatment effects are heterogenous across groups and over time, as tends to be the case with differential timing (e.g., Borusyak, Jaravel and Spiess 2021, Callaway and Sant'Anna 2021, De Chaisemartin and d'Haultfoeuille 2020, Goodman-Bacon 2021, Sun and Abraham 2021).


In the FD models shown in Table 3, both the binary and count versions of our treatment variable are significant at the 95\% confidence level. We also note that the coefficient estimates and significance levels do not change substantially after including a full set of time-varying covariates.


```{r }
# First difference (FD) regression
cat(paste("+",paste0(covars,"_diff")))

m1.fd <-  feols(police_shoot_fe_diff ~ restrictive_aele_diff | year, vcov = ~ state, data = jd.df) 
m1b.fd <-  feols(police_shoot_fe_diff ~ restrictive_aele_diff + homicide_rate_ucr_diff + property_crime_rate_ucr_diff + unemp_rate_ukcpr_diff + poverty_rate_ukcpr_diff + population_ukcpr_log_diff| year, vcov = ~ state, data = jd.df)  

m2.fd <-  feols(police_shoot_fe_diff ~ restrictive_count_aele_diff| year, vcov = ~ state,  data = jd.df) 
m2b.fd <-  feols(police_shoot_fe_diff ~ restrictive_count_aele_diff + homicide_rate_ucr_diff + property_crime_rate_ucr_diff + unemp_rate_ukcpr_diff + poverty_rate_ukcpr_diff + population_ukcpr_log_diff| year, vcov = ~ state, data = jd.df) 

```


```{r, include=TRUE, results='asis'}
#variable.labels
setFixest_dict(c(police_shoot_fe_diff = "Police shooting deaths",
restrictive_aele_diff= "Restrictive decision",
restrictive_count_aele_diff = "Restrictive decision count",
homicide_rate_ucr_diff = "State homicide rate",   
property_crime_rate_ucr_diff= "State property crime rate",
unemp_rate_ukcpr_diff = "State unemployment rate",
poverty_rate_ukcpr_diff = "State poverty rate",
population_ukcpr_log_diff = "State population (logged)" ))

fixest::etable(list(m1.fd, m1b.fd, m2.fd, m2b.fd),
               title= "First Difference estimates of the effect of restrictive judicial decisions on police shooting deaths",
               digits= 3,
               order= c("Restr*"),tex=TRUE)
```


```{r}
fixest::etable(list(m1.fd, m1b.fd, m2.fd, m2b.fd), tex=TRUE,replace = TRUE, file="results1_fd.tex")

```


\newpage{}
In the TWFE models shown in Table 4, the coefficients on our independent variables are not significant. To be sure, we do not wish to make too much of these results; we cannot be sure whether the TWFE results are null because of conceptual and theoretical concerns or because the TWFE model is a biased estimator in the case of our empirics.

```{r }
#Two-way fixed effects (TWFE) regression

m1.twfe <-  feols(police_shoot_fe ~ restrictive_aele | state + year, vcov = ~ state, data = jd.df) 
m1b.twfe <-  feols(police_shoot_fe ~ restrictive_aele + homicide_rate_ucr + property_crime_rate_ucr + unemp_rate_ukcpr + poverty_rate_ukcpr + population_ukcpr_log | state + year, vcov = ~ state, data = jd.df) 
m2.twfe <-  feols(police_shoot_fe ~ restrictive_count_aele | state + year, vcov = ~ state,  data = jd.df) 
m2b.twfe <-  feols(police_shoot_fe ~ restrictive_count_aele + homicide_rate_ucr + property_crime_rate_ucr + unemp_rate_ukcpr + poverty_rate_ukcpr + population_ukcpr_log | state + year, vcov = ~ state, data = jd.df)



```


```{r, include=TRUE, results='asis'}
setFixest_dict(c(police_shoot_fe = "Police shooting deaths",
restrictive_aele= "Restrictive decision",
restrictive_count_aele = "Restrictive decision count",
homicide_rate_ucr = "State homicide rate",   
property_crime_rate_ucr= "State property crime rate",
unemp_rate_ukcpr = "State unemployment rate",
poverty_rate_ukcpr = "State poverty rate",
population_ukcpr_log = "State population (logged)" ))

fixest::etable(list(m1.twfe, m1b.twfe,m2.twfe, m2b.twfe),
               title= "Two-way fixed effect estimates of the effect of restrictive judicial decisions on police shooting deaths",
               digits= 3,
               placement = "H",
               order= c("Restr*"),tex=TRUE)
```

\newpage{}
Figure 5 compares the $DID_M$, FD and TWFE estimates. We should keep in mind that the TWFE regression employs OLS with count dependent variables that are rare events. A Poisson model is preferable over OLS in this situation, but would not be comparable with $DID_M$, which is linear regression. We include TWFE here only because it is the most common DID estimator. $DID_M$ and FD are conceptually and analytically similar, since $DID_M$ also relies on first differences of the treatment and outcome. 

```{r}
t.FD <- rbind(
m1.fd %>% broom::tidy() %>% filter(term== "restrictive_aele_diff") %>% dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="bivariate"),
m1b.fd %>% broom::tidy() %>% filter(term== "restrictive_aele_diff")  %>% dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="with covariates"),
m2.fd %>% broom::tidy() %>% filter(term== "restrictive_count_aele_diff") %>% dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="bivariate"),
m2b.fd %>% broom::tidy() %>% filter(term== "restrictive_count_aele_diff") %>%   dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="with covariates")) %>%  select(estimate, std.error,p.value, IDV, DV, model) %>% dplyr::mutate(estimator="FD")

t.TWFE <- rbind(
m1.twfe %>% broom::tidy() %>% dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="bivariate"),
m1b.twfe %>% broom::tidy() %>% filter(term== "restrictive_aele") %>%  dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="with covariates"),
m2.twfe %>% broom::tidy() %>% dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="bivariate"),
m2b.twfe %>% broom::tidy() %>% filter(term== "restrictive_count_aele") %>%   dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="with covariates")) %>%  select(estimate, std.error,p.value, IDV, DV, model) %>% dplyr::mutate(estimator="TWFE")

t.DIDM <- rbind(
readxl::read_xlsx("table2.xlsx")[1,] %>% dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="bivariate"),
readxl::read_xlsx("table2.xlsx")[2,] %>% dplyr::mutate(IDV= "Restrictive decision", DV= "Police shooting deaths",model="with covariates"),
readxl::read_xlsx("table2.xlsx")[3,] %>% dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="bivariate"),
readxl::read_xlsx("table2.xlsx")[4,] %>% dplyr::mutate(IDV= "Restrictive decision count", DV= "Police shooting deaths",model="with covariates")) %>%  select(estimate, std.error=se,p.value=pval, IDV, DV, model) %>% dplyr::mutate(estimator="DID_M")


all.m <- rbind(t.FD, t.TWFE, t.DIDM) %>% dplyr::mutate(model=factor(model,levels=c("bivariate","with covariates")),DV=factor(DV,levels=c("Police shooting deaths")),IDV=factor(IDV,levels=c("Restrictive decision","Restrictive decision count")),estimator=factor(estimator, levels=c("DID_M","FD","TWFE")))

```

**Figure 5**: Comparison of estimators
```{r, all_models,include=TRUE}
ggplot2::ggplot(all.m, aes(x=estimator, y=estimate,shape=model)) +
    geom_point(aes(),position=position_dodge(0.5)) + 
    geom_errorbar(aes(ymin=estimate-std.error*1.96, ymax=estimate+std.error*1.96), width=0.1,position=position_dodge(0.5)) +
    geom_hline(yintercept=0, linetype="dashed",lwd=0.2) +
    scale_y_continuous(position="right") +
    facet_grid(IDV ~ DV,switch="y") + xlab("") + 
    ylab("estimate") + theme_bw() +
    theme(legend.position="bottom")

```




## Alternative operationalisation of the independent variable

The coding of the independent variable *restrictive decision* invokes an implicit assumption of no precedence; if no decision is issued in a given state-year, that state-year is coded 0. In contrast to this assumption, it is possible that judicial decisions establish a legal precedence, or status quo, which carries-over from year-to-year in the absence of a contrary decision. In order to probe our assumption of no precedence, this section presents the $DID_M$ estimates using an alternative operationalisation of the independent variable which assumes judicial decisions do establish legal precedence.

*Restrictive status quo* is a binary variable which is coded 1 in the first year a restrictive judicial decision is issued and remains coded 1 until a non-restrictive decision is issued. It is coded 0 when a non-restrictive decision is issued and remains coded 0 until the next restrictive decision. In this operationalisation, we assume that a judicial decision establishes a status quo which remains in place until the next contradictory decision. The first difference of this variable indicates whether the status quo judicial decision in a given state-year differs from the status quo in the previous year.^[We assume that all states enter the data set in a non-restrictive status quo, since the first restrictive decision is issued in 2005. *Restrictive status quo* (and its first difference) is coded 0 in the years prior to a state's first judicial decision.] 

Mechanically, this alternative operationalisation will result in fewer switches in treatment status, since treatment status carries over in years with no decision. We prefer an operationalisation which captures as much variation as possible, thus we report the results using switches in *restrictive decision* in the manuscript. We present results with the alternative operationalisation here. 

In Table 5, we see a statistically significant effect of a switch in a restrictive status quo, and the magnitude of the coefficient is similar to that using the original coding of the independent variable. The estimated effect is not statistically significant after the inclusion of covariates. Overall, this alternative operationalisation yields similar results, although the results are not as robust, possibly due to the lower number of observations used in estimation. Relaxing our assumption of no legal precedence, and allowing for fewer switches, thus does not substantively affect our conclusions. 


```{r, include=TRUE, results="asis"}
# import results from Stata

t4 <- readxl::read_xlsx("table4.xlsx") %>% mutate(estimate= ifelse(pval<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimate,se,pval,N) %>% data.frame(row.names=c("m3: Restrictive status quo", "m3b: with covariates"))

t4 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive status quo on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r") %>%
  kableExtra::kable_styling(latex_options = "HOLD_position")
```

Notes: This table reports $DID_{M}$ estimates: the average treatment effect among states that switch treatment status, at the year when the switch occurs. Model 3 uses a binary treatment variable, coded 1 in the first year a restrictive judicial decision is issued and remaining coded 1 until a non-restrictive decision is issued. Model 3b includes the following covariates: homicide rate and property crime rate. The dependent variable is the count of police shooting deaths in a state-year. Standard errors are calculated via bootstrapping (200 repetitions) and are clustered by state. Observations denotes number of observations used in the estimation of effect: the number of first differences of the outcome and of the treatment* indicates significance at the 95% confidence level.


```{r}
# First difference (FD) regression
m3.fd <-  feols(police_shoot_fe_diff ~ status_quo_diff | year, vcov = ~ state, data = jd.df) 
m3b.fd <-  feols(police_shoot_fe_diff ~ status_quo_diff  + homicide_rate_ucr_diff + property_crime_rate_ucr_diff + unemp_rate_ukcpr_diff + poverty_rate_ukcpr_diff + population_ukcpr_log_diff| year, vcov = ~ state, data = jd.df)  

# Two-way fixed effects (TWFE) regression
m3.twfe <-  feols(police_shoot_fe ~ status_quo | state + year, vcov = ~ state, data = jd.df) 
m3b.twfe <-  feols(police_shoot_fe ~ status_quo + homicide_rate_ucr + property_crime_rate_ucr + unemp_rate_ukcpr + poverty_rate_ukcpr + population_ukcpr_log | state + year, vcov = ~ state, data = jd.df) 

```

```{r}

t.DIDM.alt <- rbind(
readxl::read_xlsx("table4.xlsx")[1,] %>% dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="bivariate"),
readxl::read_xlsx("table4.xlsx")[2,] %>% dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="with covariates")) %>%  select(estimate, std.error=se,p.value=pval, IDV, DV, model)

all.m.alt <- rbind(
t.DIDM.alt,
m3.fd %>% broom::tidy() %>% filter(term== "status_quo_diff") %>% dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="bivariate") %>% select(- c(term, statistic)),
m3b.fd %>% broom::tidy() %>% filter(term== "status_quo_diff")  %>% dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="with covariates") %>% select(- c(term, statistic)),
m3.twfe %>% broom::tidy() %>% filter(term== "status_quo") %>% dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="bivariate") %>% select(- c(term, statistic)),
m3b.twfe %>% broom::tidy() %>% filter(term== "status_quo")   %>% 
dplyr::mutate(IDV= "Restrictive status quo", DV= "Police shooting deaths",model="with covariates") %>% select(- c(term, statistic))) %>% dplyr::mutate(model=factor(model,levels=c("bivariate","with covariates")),DV=factor(DV,levels=c("Police shooting deaths")),IDV=factor(IDV,levels=c("Restrictive status quo")),estimator=factor(c(rep("DID_M",2),rep("FD",2),rep("TWFE",2)), levels=c("DID_M","FD","TWFE")))

```

Figure 6 shows the FD and TWFE estimates of the alternative operationalisation of the independent variable, alongside the $DID_M$ estimates. As with the original binary and count versions of the independent variable, the FD models show a statistically significant effect of restrictive status quo which is robust to the inclusion of a full set of covariates, while the results are not statistically significant in the TWFE models. 

**Figure 6**: Comparison of estimators
```{r, all_models_alt,include=TRUE, fig.height = 3 }
ggplot2::ggplot(all.m.alt, aes(x=estimator, y=estimate,shape=model)) +
    geom_point(aes(),position=position_dodge(0.5)) + 
    geom_errorbar(aes(ymin=estimate-std.error*1.96, ymax=estimate+std.error*1.96), width=0.1,position=position_dodge(0.5)) +
    geom_hline(yintercept=0, linetype="dashed",lwd=0.2) +
    scale_y_continuous(position="right") +
    facet_grid(IDV ~ DV,switch="y") + xlab("") + 
    ylab("estimate") + theme_bw() +
    theme(legend.position="bottom")

```


## Summary of all results

Table 6 shows the estimated effects across all estimators ($DID_M$, FD and TWFE), all operationalisations of the treatment variable (binary, count and status quo), in models with and without covariates. 


```{r, include=TRUE, results='asis'}
t4 <- rbind(all.m, 
            all.m.alt %>% select(names(all.m))) %>% 
dplyr::mutate(estimate= ifelse(p.value<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimator, IDV, DV, model, estimate,std.error,p.value) %>% arrange(estimator,DV,IDV,model)

t4 %>% 
kableExtra::kbl(caption="Comparison of all estimators across all specifications",
digits= 3,
format= "latex",
col.names = c("Estimator","Treatment","Outcome","Model","Estimate","Std. error","P-value"),
                  align="l") %>%
  kableExtra::kable_styling(latex_options = "HOLD_position")

```
Note: * indicates significance at the 95% confidence level.




```{r}


save.image("JD_results.RData")


```
# Robustness checks
## Standard errors clustered on circuit

We present the main results in Table 2, with standard errors clustered on state. However, we are aware that clustering on the level of the circuit court would be appropriate given that our treatment is on the level of the circuit. Our concern with this however, is that this would mean having few clusters. We consulted Clément de Chaisemartin on this issue, one of the creators of the $DID_{M}$ estimator (De Chaisemartin and d’Haultfoeuille 2020). He recommended that we cluster at the level of the state. Doing so means that our standard errors are conditional on the realized treatments, an implication with which we are comfortable. Importantly, however, the standard errors in our models with state-level clustered errors account for potential serially-correlated state-level shocks on the outcome. Although we prefer to report results with state-level clustered standard errors, we also show the main results with circuit-level standard errors here. As expected, the results are not statistically significant, likely due to having a relatively low number of clusters and a lower number of effective observations. 


**Table 7** replicates the main results reported in Table 2, with standard errors clustered on circuit. 
```{r, include=TRUE, results='asis'}

# import results from Stata

t7 <- readxl::read_xlsx("table7.xlsx") %>% mutate(estimate= ifelse(pval<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimate,se,pval,N) %>% data.frame(row.names=c("m1: Restrictive decision", "m1b: with covariates","m2: Restrictive decision count","m2b: with covariates"))

t7 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r") %>%
  kableExtra::kable_styling(latex_options = "hold_position")
```
Notes: This table reports $DID_{M}$ estimates: the average treatment effect among states that switch treatment status, at the year when the switch occurs. Model 1 uses a binary treatment variable, indicating at least one restrictive judicial decision on ECD-use in a state-year. In model 2 the treatment variable is the count of restrictive decisions in a state-year. Models 1b and 2b include the following covariates: homicide rate and property crime rate. The dependent variable in all models is the count of police shooting deaths in a state-year. Standard errors are calculated via bootstrapping (200 repetitions) and are clustered by circuit. Observations denotes number of observations used in the estimation of effect: the number of first differences of the outcome and of the treatment. * indicates significance at the 95% confidence level.



## Excluding corrections cases

The data in the main article include AELE cases which are listed as 'corrections and confinement.' These cases include incidents which occurred in prisons as well as incidents where TASERS were used by police in post-arrest, on-the-street encounters. Theoretically, we expect these rulings to influence police behavior regarding use of force just as much as the pre-arrest cases. In this section, however, we replicate the main results omitting the corrections and confinements cases. As shown in Table 8, excluding corrections cases yields a similar estimated effect, although the effect is not robust to the inclusion of control variables. This could be due to the lower number of observations used in estimation. 

**Table 8** replicates the main results reported in Table 2, excluding corrections cases. 
```{r, include=TRUE, results='asis'}

# import results from Stata

t8 <- readxl::read_xlsx("table8.xlsx") %>% mutate(estimate= ifelse(pval<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimate,se,pval,N) %>% data.frame(row.names=c("m1: Restrictive decision", "m1b: with covariates","m2: Restrictive decision count","m2b: with covariates"))

t8 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r") %>%
  kableExtra::kable_styling(latex_options = "hold_position")
```
Notes: This table reports $DID_{M}$ estimates: the average treatment effect among states that switch treatment status, at the year when the switch occurs. Model 1 uses a binary treatment variable, indicating at least one restrictive judicial decision on ECD-use in a state-year. In model 2 the treatment variable is the count of restrictive decisions in a state-year. Models 1b and 2b include the following covariates: homicide rate and property crime rate. The dependent variable in all models is the count of police shooting deaths in a state-year. Standard errors are calculated via bootstrapping (200 repetitions) and are clustered by state. Observations denotes number of observations used in the estimation of effect: the number of first differences of the outcome and of the treatment. * indicates significance at the 95% confidence level.


## Recoding a single case

We noticed that in the AELE case summaries, \emph{Roberts v. Manigold}, #06-2039, 240 Fed. Appx.s 675, 2007 U.S. App. Lexis 14514 (Unpub. 6th Cir. 2007), is listed twice. In its first reference, it is not denoted as Restrictive; in its second reference, it is denoted as Restrictive. We originally coded the case in our data as Restrictive. Here, we show the robustness of our results to the re-coding \emph{Roberts v. Manigold} as non-restrictive.


**Table 9** replicates the main results reported in Table 2, re-coding \emph{Roberts v. Manigold} as non-restrictive. 
```{r, include=TRUE, results='asis'}


# import results from Stata

t9 <- readxl::read_xlsx("table9.xlsx") %>% mutate(estimate= ifelse(pval<0.05,paste0(round(estimate,3),"*"),round(estimate,3))) %>% select(estimate,se,pval,N) %>% data.frame(row.names=c("m1: Restrictive decision", "m1b: with covariates","m2: Restrictive decision count","m2b: with covariates"))

t9 %>%
kableExtra::kbl(caption="$DID_{M}$ estimates of the effect of restrictive judicial decisions on police shooting deaths",
digits= 3,
format= "latex",
col.names = c("Estimate","Standard error","P-value","Observations"),
                  align="r")  %>%
  kableExtra::kable_styling(latex_options = "hold_position")
```
Notes: This table reports $DID_{M}$ estimates: the average treatment effect among states that switch treatment status, at the year when the switch occurs. Model 1 uses a binary treatment variable, indicating at least one restrictive judicial decision on ECD-use in a state-year. In model 2 the treatment variable is the count of restrictive decisions in a state-year. Models 1b and 2b include the following covariates: homicide rate and property crime rate. The dependent variable in all models is the count of police shooting deaths in a state-year. Standard errors are calculated via bootstrapping (200 repetitions) and are clustered by state. Observations denotes number of observations used in the estimation of effect: the number of first differences of the outcome and of the treatment. * indicates significance at the 95% confidence level.







# Examples of restrictive and non-restrictive decisions

Here, we show a few examples of restrictive and non-restrictive circuit court decisions related to Electronic Control Devices (ECDs). The summary text is taken directly from directly from the AELE website.^[Americans for Effective Law Enforcement (AELE), "Electronic Control Weapons AELE Case Summaries", August 6, 2019, Distributed by AELE, https://www.aele.org/law/Digests/ECWcases.html#1st_Circuit. Accessed on March 8, 2023.]

## Restrictive decisions
**\emph{Yates v. Terry}**, #15-1555, 2016 U.S. App. Lexis 5903 (4th Cir.).

A motorist pulled over for a traffic stop claimed that an officer fired a Taser in the dart mode at him for no apparent reason, activating it three times. He was charged with an excessive noise violation, not having a driver's license in his possession, and disorderly conduct, with all charges later dropped. A federal appeals court denied qualified immunity to the officer on an excessive force claim. It noted that the plaintiff's alleged offenses were nonviolent minor traffic offenses, there was no evidence establishing that he was a danger to the officer at any time during the incident, and he was not attempting to flee or resist. Prior precedent put the officer on fair notice that the force he used against the arrestee under the alleged facts of the case was unconstitutionally excessive. 

**\emph{Smith v. Appledorn}**, #11-2966, 2013 U.S. Dist. Lexis 16138 (D. Minn.).

A man got into a street fight with someone who had grabbed his girlfriend's buttocks. The other man's friends joined in and beat him up. A police officer broke up the fight, using a chemical spray, and told the man to go home. In a nearby parking lot, the fight started up again between the man's assailants and a male friend of his. The officer then came by and allegedly pushed that man to the ground. When the man got up and questioned why the officer had done that, the officer allegedly pushed him again and used Taser in the stun mode on his buttocks, after which he was handcuffed and arrested. A court found that the officer had no specific or articulable basis to seize the man, who was complying with orders to go home, was not behaving violently or aggressively, and was not committing any crime. Under these circumstances, assuming that the plaintiff's version of the incident was true, the use of the Taser, or any force at all, would have been unreasonable. The defendant officer was not entitled to summary judgment. 

**\emph{Thomas v. Plummer}**, #11-3165, 2012 U.S. App. Lexis 14843, 2012 Fed. App. 0770N (Unpub. 6th Cir.).

A female passenger in a car stopped for a hit-and-run violation got out of the vehicle and was repeatedly ordered to get on the ground. She had stepped out of the car because officers were arresting the driver on a previous felony warrant and she had refused orders to get back in the car. After she got on her knees and placed her hands above her head, an officer discharged his Taser in the dart mode into her back. The officer who used the Taser was not entitled to qualified immunity because using the Taser under these circumstances against a previously disobedient suspect who had stopped resisting constituted excessive force under clearly established law. The woman posed no threat at the time, and the accused offense of obstructing official business by initially refusing to obey orders to get on the ground was not a particularly serious offense. 

## Non-restrictive decisions

**\emph{Simmonds v. Genesee County}**, #10-1470, 2012 U.S. App. Lexis 12347 (6th Cir.).

Police responded to 911 calls indicating that a man had threatened to kill his ex-girlfriend's parents. They believed that he might be armed with a gun. In a heavily wooded area, he ignored orders to show his hands, allegedly yelled that he had a gun, and had been drinking and acting in a mentally disturbed manner. An officer's attempt to use a Taser in the dart mode on him failed because of a heavy coat he had on. He brandished a silver object which turned out to be a phone. An officer who believed it might have been a gun shot and killed him. Under these circumstances, the use of deadly force was justified, despite the fact that, with hindsight, it turned out that he was unarmed. ^[In the district court decision Simmonds v. County of Genesee, #09-12286, 2010 U.S. Dist. Lexis 28391 (E.D. Mich.), the court ruled that the use of the Taser was objectively reasonable. The motorist refused to either show his hands or obey commands to exit his vehicle. The officer who used the Taser had been informed that the suspect had threatened to kill a former girlfriend and was armed, and only used the Taser after repeated verbal commands failed. The officer was entitled to qualified immunity for the use of the Taser.] 

**\emph{Lindsay v. Kiernan}**, #09-55652, 378 Fed. Appx. 606, 2010 U.S. App. Lexis 8910 (Unpub 9th Cir.). 

Officers responded to a convenience store where they encountered a six foot, 220 lb. man who was obviously intoxicated, angry, and belligerent. He was ordered to leave the area in the taxi that he had arrived in. When he refused and tried to enter the store shouting an obscenity, an officer said, "Okay, you want the Taser?" The man responded, "I don't mind it." The Taser was deployed in the dart mode for 5 seconds, which took him down. Another 4-second cycle was used to assist in handcuffing the man. In the suit that followed, the U.S. District Court granted a Summary Judgment for the defendant officer. In affirming, the Ninth Circuit appellate panel wrote that the plaintiff, "while heavily intoxicated, actively resisted the officers repeated verbal commands to leave the gas station, broke free from [the officer's] grasp, and barreled through five officers to confront the young female convenience store clerk." He ignored a warning that he would be Tasered. "Such conduct qualifies as more than minor resistance. In addition, a reasonable officer could have concluded that [the plaintiff] continued to actively resist arrest by attempting to get up after the first Taser shot, despite officers' commands to stay down and submit to arrest." 

**\emph{Sanders v. City of Dothan}**, #09-16472, 409 Fed. Appx. 285, 2011 U.S. App. Lexis 1129 (Unpub. 11th Cir.), affirming Sanders v. City of Dothan, #1:07-cv-008, 671 F.Supp.2d 1263 (M.D. Ala. 2009).

Officers believed that an arrestee placed in their car after a chase had cocaine both in his car and in his beard. A Taser was used on the arrestee to try to get him to open his mouth and lift his tongue to reveal if he had any drugs there. The arrestee subsequently died from cocaine intoxication. There was no proof that the officers were liable for the death as they had not been aware the arrestee had swallowed large amounts of cocaine. The officers were entitled to qualified immunity for using the Taser on the arrestee under these circumstances. The officers had a legitimate objective in trying to get the arrestee to open his mouth, and there was no clearly established case law prohibiting their actions. 


\newpage
# References

Borusyak, Kirill, Xavier Jaravel and Jann Spiess. 2021. "Revisiting Event Study Designs: Robust and Efficient Estimation." arXiv preprint arXiv:2108.12419 .

Callaway, Brantly and Pedro HC Sant'Anna. 2021. "Difference-in-Differences with Multiple Time Periods." \emph{Journal of Econometrics} 225(2):200–230.

De Chaisemartin, Clément and Xavier d'Haultfoeuille. 2020. "Two-way Fixed Effects Estimators with Heterogeneous Treatment Effects." \emph{American Economic Review} 110(9):2964–96.

Goodman-Bacon, Andrew. 2021. "Difference-in-differences with Variation in Treatment Timing." \emph{Journal of Econometrics} 225(2):254–277.

Liu, Licheng, Ye Wang and Yiqing Xu. 2022. "A Practical Guide to Counterfactual Estimators for Causal Inference with Time-Series Cross-Sectional Data." \emph{American Journal of Political Science} (Forthcoming).

Schwartz, J. C. (2021). Qualified Immunity's Boldest Lie. \emph{University of Chicago Law Review}, 88:605–684.

Sun, Liyang and Sarah Abraham. 2021. "Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects." \emph{Journal of Econometrics} 225(2):175–199.
